The field of human-computer interfaces is a growing one. From the evolution of the keyboard to the invention of the mouse, newer and better means for interacting with computers have always been sought. One seeming constant in past and current interfaces has been the requirement for human hands, digits, or other physical appendages to manipulate some sort of implement to interact with a computer. Keyboards, mice, light pens, track balls and other human-computer interfaces all require human interaction and physical manipulation of at least some part of the interface.
Unfortunately, all of the above interfaces are inaccessible to people who are incapable of physically manipulating such interfaces. While some speech driven interfaces are available, these are far from ideal. Navigating a graphical user interface using speech can be quite difficult and “training” the software to recognize the users voice patterns and commands is time consuming.
For a vision based interface to be fully operational, it has to be able to do two tasks. First, it should be able to track a human face both robustly—with respect to head motion, and precisely—with subpixel precision, so that its position can be converted to a position of a cursor or another virtual object in a 2D screen. Second, the interface should be able to detect a facial expression event. This facial expression event would be used by a user to send a binary “on/off” command to the system, a command that is analogous to a mouse “click” event.
The recently developed Nouse “Use your nose as a mouse” tracking technology made it possible to track a liberally unconstrained head motion with subpixel precision, thereby offering a solution to the first of these two tasks. For more information on the Nouse technology, see Gorodnichy (Gorodnichy, “On Importance of nose for face tracking”, Proc. IEEE Intern. Conf. On Automatic Face and Gesture Recognition (FG'2002), Washington D.C., 2002), Gorodnichy et al (Gorodnichy, Malik, and Roth, “Nouse—A New Technology for Hands-free Games and Interfaces”, Proc. Intern. Conf. On Vision Interface (VI'2002), Calgary, 2002), and www.perceptual-vision.com.
While head-tracking technologies such as the Nouse show promise, they still have one drawback that makes them fall short of the mouse in terms of ease of use. Some head-tracking technologies propose the sending of binary commands such as “clicking” using eye blinks. However, detecting eye based binary commands, such as blinks and double-blinks, have previously proved difficult and results have been poor. One major problem has been the difficulty in determining how to locate a person's eyes when his face moves. Given the non-static nature of the human head, this is not a trivial problem and previous attempts to solve it has met with, at best, mixed results. Such a drawback limits the usability of such head-tracking methods and technologies.
A common approach to detecting moving objects in video is based on detecting the intensity change between two consecutive frames caused by the object motion. The simplest way of detecting such a change, which will be referred to as a first order change, is to use two consecutive video frames to decide where such a change occurred. This is what has been used so far to detect blinks. However, this technique fails to detect eyes when a face moves as many candidates also appear around the face boundary as well as around the nose, mouth, and other parts of the face.
Based on the above, there is therefore a need for methods and/or devices that can be used to allow for clicking in conjunction with head-tracking technologies. Such methods and/or devices may also have other applications in the image processing field.